Transforming EHR Interactions Using Voice Assistants

Session 164, February 13, 2019

Yaa Kumah-Crystal, MD MPH

Timothy Coffman

Vanderbilt University Medical Center

Yaa Kumah-Crystal, MD MPH

Discloses the following relevant relationship with commercial

interests:

• Research partnership with Nuance Communications

• Use of NLP libraries for research and development

Conflict of Interest

Timothy Coffman

Discloses the following relevant relationship with commercial

interests:

• Research partnership with Nuance Communications

• Use of NLP libraries for research and development

Conflict of Interest

• Principles employed in creating a voice user interface for the

electronic health record

• Challenges faced in data capture

• Challenges faced in data presentation

• Assessment

• Implementation and future plans

Agenda

• Discuss the value of artificial intelligent (AI)-enabled natural

language processing and understanding technologies in the

advancement of healthcare

• Discuss the limitations of current keyboard and mouse workflows

in the EHR

• Describe the usability design considerations, information request

mapping and artificial intelligence/natural language understanding

platform built to process voice requests

• Discuss the provider cohort experience and outcomes around

satisfaction

Learning Objectives

Informal Poll

Own a smart phone

Use your smart phone voice assistant

• Siri, Google assistant, Bixby

Own a voice assistant device

• Alexa, Google home, Apple Homepod

Why Voice?

Documentation Modalities

Writing

~13

WPM

Typing

~40

WPM

Speech

~150

WPM

Why not Voice?

Macros

Physical exam

Insert normal physical exam

Vanderbilt Innovations Portfolio Research

Given

• Installed EHR

• EHR vendor has existing partnership with natural language

services provider (including dictation)

• Existing in-house innovation capacity

– Infrastructure supporting custom development

– Team in place with necessary SW development skills

– Mission to develop expertise in new ways to interact with the EHR

Decision

• Develop new software to integrate EHR with additional natural

language services

Solutions

Well really, how how hard can it be?

Oh...

Demonstration

VEVA Processing

NLP

User Query

Speech to text

Intent Fulfillment

Text to speech

Results Display

What is her

A1C?

Sally’s A1C

is 8.6

VEVA Architecture

• EHR’s FHIR implementation does not yet support all desired data

• Additional development required for each new query

• New phrasings of questions requiring updates to language model

• Microphone and speaker issues in a virtual desktop environment

• Latency

Technical Limitations

Focus on developing demonstration of capabilities supporting

• One (1) specialty clinic

• Handful of general responses

• Aligned with FHIR (DSTU2) resources

• Some EHR-proprietary services

Build in capability to extend with

• Additional capabilities for other specialties

• General responses, discovered through user feedback

• FHIR (STU3) and FHIR (R4) resources as available

• FHIR-based alternatives to proprietary services

Planning for Extension

• MVP

• Rapid Iteration

• Vanderbilt as a Design Shop

Technical Considerations

Integrating voice response involves real-time services

• Audio streaming capabilities

– Virtual desktops introduce challenges

• Automatic connections

• Latency

– Varying hardware installed at point-of-use

• Individual laptop microphones

• High-end dictation microphones

• All of the above

Infrastructure Challenges

• “While voice is a natural interface for humans, this may be the first

time we are using it as an interface to have a computer

communicate back to us like this. Different rules may apply.”

Design Considerations

• Brevity vs. Clarity

• “Blood pressure is 120 millimeters of mercury over 80

millimeters of mercury ”

• “Blood pressure is 120 over 80”

• Uncanny Valley of Words

• I ate three big red apples

• I ate red big three apples

• Quantity, Opinion, Size, Age, Shape, Color, Material,

Purpose

Design Considerations

Use Cases

• Preparing to see a patient in a provider workroom

– Large screen

– PHI private

Use Case A

• Reviewing information with patient present

– Large screen

– PHI with patient 3

party

Use Case B

• In transition in a private environment (commute)

– Small Screen (mobile)- limited visibility

– Primarily verbal navigation

– PHI private

Use Case C

• In transition in a public environment (hallway)

– Small screen (mobile)

– PHI considerations

(aloud only with headphones?)

Use Case D

Usability Study

• Setting: Vanderbilt University Medical Center

• Cohort: 14 Pediatric Endocrinology Providers

• Female (10) | Male (4)

• Mean age 39, range 29-65

Study Design

• Usability Assessment

• 6 Skills Evaluated

– Training: Hello / What can you do

– Summary

– A1C

– Weight

– Blood Pressure

– Health Maintenance

Study Design

• Value of information communicated

• Length of content communicated

• Value of graphical display

• Overall thoughts about voice interactions

Feedback

System Usability Scale: Positively Worded

Strongly Disagree

Mean 4.12 (SD .46)

3.93 (SD 1.20)

4.50 (SD .52)

3.79 (SD .70)

4.00 (SD .79)

4.43 (SD .51)

Strongly Agree

System Usability Scale: Negatively Worded

Mean 1.67 (SD .66)

Strongly Disagree

Strongly Agree

1.36 (SD .84)

1.71 (SD .83)

1.79 (SD 1.19)

1.64 (SD .74)

1.86 (SD 1.03)

• Mean System Usability Scale score 80.7 (highly usable)

• Scores of >68 are considered above average

• Summary

• Relevant medications, other problems

• Weight

• Growth chart

• Blood pressure

• Percentile information

User Feedback

Willingness to Use System

36%

64%

NoYes

4 count9 count

• Incorporation of User Feedback

• Desktop Time Comparison

• Scalability

• Alerts and Decision Support

Next Steps

• Understand and measure provider frustration with EHR

• Create the business case for optimizing their experience and

saving providers time within their EHR workflow

• Assemble a cross-functional team to build the overarching model

taking into consideration the providers’ pain points, workflow, and

information needs

• Build and iterate a voice assistant prototype while users test and

provide feedback

• Understand information theory and map queries and content to

satisfy users’ needs

Best Practices/Recommendations

An Interesting Study About Voice

Nationwide Children's Hospital in Columbus, Ohio

It looks like you have a question.

Yes

Go away Clippy!

Questions

Please remember to complete the online session evaluation